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Disclosure to Promote the Right To Information 

Whereas the Parliament of India has set out to provide a practical regime of right to 
information for citizens to secure access to information under the control of public authorities, 
in order to promote transparency and accountability in the working of every public authority, 
and whereas the attached publication of the Bureau of Indian Standards is of particular interest 
to the public, particularly disadvantaged communities and those engaged in the pursuit of 
education and knowledge, the attached public safety standard is made available to promote the 
timely dissemination of this information in an accurate manner to the public. 
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NATIONAL FOREWORD 

This Indian Standard which is identical with ISO/IEC 12119:1 994 'Information technology — Software 
packages — Quality requirements and testing' issued jointly by the International Organization for 
Standardization (ISO) and International Electrotechnical Commission (lEC) was adopted by the 
Bureau of Indian Standards on the recommendation of Software Systems, Languages and 
Methodologies Sectional Committee (LTD 33) and approval of the Electronics and Telecommunication 
Division Council. 

The text of the ISO/IEC standard has been approved as suitable for publication as Indian Standard 
without deviations. Certain conventions are, however, not identical to those used in Indian Standards. 
Attention is particularly drawn to the following: 

Wherever the words 'International Standard' appear referring to this standard, they should be 
read as 'Indian Standard'. 

CROSS REFERENCES 

The concerned technical committee has reviewed the provisions of the following standards referred 
in this adopted standard and has decided that these are acceptable for use in conjunction with this 
standard: 

ISO/IEC Guide 22 : 1996 General criteria for supplier's declaration of conformity 

ISO/IEC Guide 23 : 1982 Methods of indicating conformity with standards for third-party certification 
systems 

ISO/IEC Guide 25 : 1990 General requirements of the correspondence of certification and testing 
laboratories 

ISO/IEC Guide 28 : 1982 General rules for a model third-party certification system for products 

ISO/IEC Guide 44 ; 1985 General rules for ISO or lEC international third-party certification schemes 
for products 

ISO/IEC Guide 58 : 1993 Calibration and testing laboratory accreditation systems — General 
requirements for operation and recognition 

ISO/IEC Guide 60 : 1994 ISO/IEC Code of good practice for conformity assessment 

iSO/lEC Guide 65 : 1 996 General requirements for bodies operating product certification systems 

NOTE — ISO/IEC Guide 16 : 1978 has been now replaced by ISO/IEC Guide 60 : 1994 and ISO/IEC Guide 40 ; 1983 has 
been now replaced by ISO/IEC Guide 65 : 1996. 
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Indian Standard 

INFORMATION TECHNOLOGY — SOFTWARE 
PACKAGES — QUALITY REQUIREMENTS 

AND TESTING 



1 Scope 

This International Standard is applicable to software 
packages. Examples are text processors, spread- 
sheets, data base programs, graphics packages, 
programs for technical or scientific functions, and util- 
ity programs. 

It establishes 

- requirements for software packages (quality 
requirements); 

- instructions on how to test a software package 
against these requirements (instructions for test- 
ing, in particular for third party testing). 

It deals only with software packages as offered and de- 
livered. It does not deal with their production process 
(including activities and intermediate products, e.g. 
specifications). The quality system of a supplier is out- 
side the scope of this International Standard. 

NOTE - Some software needs additional requirements, e.g. safety- 
critical software. 

The intended users of this International Standard in- 
clude 

a) suppliers when 

1) specifying requirements for a software 
package; 

2) designing a form to describe products; 

3) assessing their own products; 

4) issuing declarations of conformity [ISO/ 
lEC Guide 22]; 

5) applying for certificates or marks of con- 
formity [ISO/IEC Guide 23]; 



b) certification bodies which may wish to estab- 
lish a third-party certification scheme (international, 
regional or national) [ISO/IEC Guides 16, 28 and 
44]; 

c) testing laboratories which will have to follow 
the instructions for testing when testing for a 
certificate or a mark of conformity [ISO/IEC Guide 

25]; 

d) accreditation bodies for accrediting certifica- 
tion bodies and testing laboratories [ISO/IEC 
Guides 40 and 58]; 

e) auditors of testing laboratories when assess- 
ing their competence [ISO/IEC Guide 58]; 

f) buyers who may 

1) compare their requirements with those de- 
scribed here; 

2) compare the requirements for the intend- 
ed work task with the information in product 
descriptions of existing products; 

3) look for certified products; 

4) check otherwise if the requirements are 
met; 

g) users who may profit from better products. 



2 Definitions 

For the purposes of this International Standard, the 
following definitions apply. Definitions from other 
standards used in this International Standard are 
reproduced in annex A for convenient reference. 
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2.1 function: The implementation of an algorithm in 
the program with which the user or the program can 
perform part or all of a work task. 

NOTES 

1 A function does not need to be callable by the user (e.g. automatic 
backup or sa\nng of data). 

2 The notion of a function here is narrower than that used in ISO 
2382-14:1978 (in the definitions of failure, fault, maintenance and relia- 
bility), but wider than those defined in ISO 2382-2 and ISO 2382-15. 

2.2 requirements document: A document contain- 
ing any combination of recommendations, require- 
ments or regulations to be met by a software package. 

NOTE - Examples are a technical or ergonomic standard, a require- 
ments list (or model requirements specification) from a group (e.g. a 
market sector, technical or user association), a law or a decree. 

2.3 product description: A document stating prop- 
erties of a software package, with the main purpose of 
helping potential buyers in the evaluation of the suit- 
ability for themselves of the product betore 
purchasing it. 

NOTE - This temn is more specific than the term "system description" 
in ISO/IEC 2382-20:1990. The purpose of the product description 
Includes that of the "cover information" in ISO 9127. The product de- 
scription is not a specification; it serves a different purpose. 

2.4 user documen^tation: The complete set of docu- 
ments, available in printed or non-printed form, that is 
provided for the application of the product and also is 
an integral part of the product. 

2.5 package documentation: The product descrip- 
tion and the user documentation. 

2.6 test case: A documented instruction for the 
tester that specifies how a function or a combination of 
functions shall or should be tested. A test case in- 
cludes detailed information on the following issues: 

- the test objective; 

- the functions to be tested; 

- the testing environment and other conditions 
(configuration details and preparatory work); 

- the test data; 

- the procedure; 

- the expected behaviour of the system. 



2.7 maintenance: That part of system maintenance 
(see A.5.2) which is concerned with modifying a soft- 
ware package. 



3 Quality requirements 

Subclauses 3.1 to 3.3 contain 

- the requirement that each software package 
has a product description and user documenta- 
tion; 

- requirements for the product description. In 
particular, there is a requirement that it shall con- 
tain specified information and that all its state- 
ments shall be testable and correct; 

- requirements for the user documentation; 

- requirements for the programs and for data, if 
any, included in the software package. 

NOTES 

1 The requirements on user documentation, programs and data 
contain many general requirements (independent from what may be 
promised in a product description), but they do not include all the 
properties that users may desire. 

2 Certain properties, for example "understandability" and "ease of 
overview" of the user documentation and of the program messages, are 
admittedly required in the view of the user. However, due to the 
difficulty of testing for them with clear-cut and reproducible results 
these are formulated currently as recomnnendations only. 

3 The requirements in 3.1 to 3.3 are arranged in the same order as 
the cfiaracteristics which appear in ISO/IEC 9126. 

A software package conforms to this International 
Standard if it complies with all the requirements in 3.1 
to 3.3. The recommendations (indicated by the use of 
the verbal form "should") are optional. 

NOTE 4 - Conformity of a product to the requirements In 3.1 to 3.3 may 
be difficult or impossible to prove. However, a test (including review of 
xk>cuments) according to clause 4 is deeiVied sufficient to provide the 
confidence required for a certificate of conformity according to ISO/IEC 
Guide 2. No formal proof is r^eeded. 



3.1 Product description 

Each software pac4cage shall have a product descrip- 
tion. 
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The product description defines the product. It is a 
part of the package documentation of the product. It 
provides Information on the user documentation, the 
programs and, if any, the data. 

The main purposes of the product description are 

- to help the user or potential buyer in their eval- 
uation of the suitability of the product for them- 
selves. To this extent it is also sales Information; 

- to serve as a basis for testing (see clause 4). 

The product description shall be available for people 
interested in the product. 

3.1.1 General requirements on contents 

The product description should be sufficiently under- 
standable, complete and easy to overview for helping 
potential buyers in the evaluation of the suitability for 
themselves of the product before purchasing it. 

It shall be free from Internal inconsistencies. Each term 
should have the same meaning everywhere. 

The statements of the product description shall be 
testable and correct. 

NOTE - This requirement extends to the statements in external 
requirements ckxximents invcsked, if arvy (see 3.1 .2 e). 

The following subclauses 3.1.2 to 3.1 ,8 specify what 
the product description shall or should include. It may 
include additional statements on the product. 

3.1.2 Identifications and indications 

a) Identification of the product description 

The product description shall possess a unique docu- 
ment identification. It may be named differently from 
"product description", for example: "Functional De- 
scription", "Product Information", "Product Sheet". 

b) Identification of the product 

The product description shall identify the product. 
The product identification shall have at least the prod- 
uct name and a version or date. If there are two or 
more variants mentioned In the product description, 
then each variant shall have at least the product name, 
a variant name and a version or date. 



c) Supplier 

The product description shall contain the name and 
address of at least one supplier. 

NOTE - The name and address need not be printed; ttie rubber stamp 
of a dealer is sufficient. 

d) Work task 

The product description shall identify the intended 
work tasks that can be performed with the product. 

e) Conformity to requirements documents 

The product description may refer to requirements 
documents to which the product conforms, in which 
case the relevant editions shall be identified. 

f) Required system 

The required system (hardware, software and their 
configuration) for putting the product into use shall be 
specified, Including manufacturer names and type 
Identifiers of all parts, for example 

- processing unit including co-processors; 

- main memory size; 

- types and sizes of peripheral storage; 

- extension cards; 

- input and output equipment; 

- network environment; 

- system software and other software. 

Different required systems may be specified, e.g. for 
different work tasks, different boundary values or dif- 
ferent efficiency requirements. 

The statement "(or any other ..., if compatible)" may 
appear in a product description if previously a particu- 
lar hardware or software product has been identified. 
The statement "or an updated version if compatible" 
may appear if previously a version of a product has 
been identified. The statement "from version X to at 
least version Y" may appear; "from version X" shall not 
appear. 

NOTE - A statement "from version X" could later become false by the 
appearance of a version X+3 with which the software package would 
fail to operate. 
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g) Interfaces to other products 

If the product description makes reference to inter- 
faces to other products, the Interfaces or products 
shall be identified. 

h) Items to be delivered 

Every physical component of the supplied product 
shall be identified, in particular all printed documents 
and all data media. 

The form of the supplied programs shall be stated, 
e.g. source programs, object modules or load mod- 
ules. 

NOTE - Media formats (e.g. diskette formats) need not be indicated 
because \he set of possible formats is determined by the required 
system (see 3.1 .2 f). 

i) Installation 

It shall be stated whether or not the product installa- 
tion can be carried out by the user. 

j) Support 

It shall be stated whether or not support for operating 
the product is offered. 

k) f\/laintenance 

It shall be stated whether or not maintenance is of- 
fered. If maintenance is offered, it shall be stated what 
is specifically included. 

3.1.3 Statements on functionality 

a) Overview of functions 

The product description shall provide an overview of 
the user-callable functions of the product, the neces- 
sary data and the facilities offered. 

It shall be clearly stated for each function mentioned 
(especially for an option or variant) whether it is a part 

- of the product; 

- of a product extension fully described in the 
product description; 

- of a product extension referenced in the prod- 
uct description; 



NOTE - Not every user-callable function need be mentioned, and not 
every detail on Inow a function is called need be given. 

b) Boundary values 

If the usage of the product is limited by product speci- 
fic boundary values, these shall be provided. Exam- 
ples are 

- minimum or maximum values; 

- lengths of keys; 



— maxiinuro number uf reuords in a file; 



- maximum number of search. criteria; 

- minimum sample size. 

In the case when it is not possible to provide fixed 
boundary values (when, for example, they depend 
upon the application type or upon the input data), 
then the limitations shall be stated. Permissible combi- 
nations of values may be provided and reference shall 
be made to more specific information in the user docu- 
mentation. 

c) Security 

The product description should include information 
on means, if provided, for preventing unauthorized 

and data. 

3.1.4 Statements on reliability 

The product description shall include information on 
data saving procedures. 

NOTE - it is sufficient to state, for example, that backup is possible by 
functions of tfie operating system. 

Additional product properties should be described 
that ensure the functional capability of the product. 
Examples are 

- checks whether input is plausible; 

- protection against serious consequences of a 
user mistake; 

- error recovery. 



of a supplement without guarantee. 
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3.1.5 Statements on usability 

a) User interface 

The type of user interface shall be named, for exam- 
ple, command line, menu, windows, function key, 
help function. 

b) Required knowledge 

Specific knowledge required for the application of the 
product shall be specified. Examples are 

- knowledge of a technical area; 

- knowledge of an operating system; 

- knowledge obtainable by special training; 

- knowledge of a language other than that in 
which the product description is written. 

All the natural languages of the user documentation 
and of the user interface (including error messages 
and visible data) shall be stated, both of the software 
package itself and of all other products mentioned in 
the product description. 

NOTE - This requkement exceeds the one in ISO 9127:1988, 6.1 .7, 
where the mention of the languages used is optional. 

c) Adaptation to the user's needs 

If the product can be adapted by the .user, then the 
tools for this adaptation and the conditions of their use 
shall be identified. Examples are 

- changing of parameters; 

- changing of algorithms for computation; 

- assignments to function keys. 

d) Protection against copyright infringement 

If technical protection against copyright infringement 
can hamper usability, then this protection shall be 
stated. Examples are 

- technical protection against copying; 

- programmed expiry dates for usage; 

- interactive reminders to pay for copies. 



e) Efficiency of usage and user satisfaction 

The product description may include data on efficien- 
cy of usage and user satisfaction. 

NOTE - Such data may follow the guidance of ISO 9241-1 1 . 

3.1.6 Statements on efficiency 

The product description may include data on the time 
behavior of the product such as response times and 
throughput rates for given functions under stated 
conditions (for example the system configurations and 
load profiles). 

3.1.7 Statements on maintainability 

The product description may contain statements on 
maintainability. 

3.1.8 Statements on portability 

The product description may contain statements on 
portability. 



3.2 User documentation 

3.2.1 Completeness 

The user documentation shall contain the information 
necessary for the use of the product. All the functions 
stated in the product description and all user-callable 
functions in the program shall be completely de- 
scribed in the user documentation. 

All boundary values given in the product description 
shall be repeated in the user documentation. 

If installation can be carried out by the user, the user 
documentation shall include an installation manual 
containing all necessary information (see 3.3.1 a). The 
installation manual should state the minimum and 
maximum sizes of the files once installed. 

If maintenance can be carried out by the user, the user 
documentation shall include a program maintenance 
manual containing all the information that is necessary 
for the kind of maintenance concerned. 

3.2.2 Correctness 

All information in the user documentation shall be cor- 
rect. In addition, its presentation should be free from 
ambiguities and errors. 
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3.2.3 Consistency 

The documents of the user documentation shall be 
free from contradiction within themselves, with each 
other and with the product description. Each term 
should have the same meaning everywhere. 

NOTE - Consistency with programs and data is dealt with in 3.3.1 .d. 

3.2.4 Understandability 

The user documentation should be understandable 
by the user population that normally performs the 
stated work task, for example by using an appropriate 
selection of terms, graphical exhibits, detailed ex- 
planations and by referencing useful information sour- 
ces. 

3.2.5 Ease of overview 

The user documentation should be easy to overview 
so that relationships are recognizable. 

Every document should have a table of contents and 
an index. 

If a document is. not provided in printed form the pro- 
cedure for printing it should be indicated. 

3.3 Programs and data 
3.3.1 Functionality 

a) Installation 

If installation can be carried out by the user, it shall be 
possible to install the programs successfully by 
following the information in the installation manual. 
Each of the required systems given in the product de- 
scription shall be sufficient for installing the programs. 

Following installation it shall be recognizable whether 
or not the programs can function, for example using 
supplied test cases or by self-testing with correspond- 
ing messages. 

b) Presence of functions 

All functions mentioned in the user documentation 
shall be executable in the form given in the user docu- 
mentation with the corresponding facilities, properties 
and data, and within the boundary values given there. 

NOTE - Since ail functions mentioned in the product description shall 
also appear in the user documentation, it follows that they too shall be 
executable. 



c) Correctness 

The programs and data shall correspond to all the 
statements in the product description and the user 
documentation. The functions shall be executed in a 
correct manner for the work task. In particular, pro- 
grams and data shall comply with all the requirements 
in any requirements document referenced by the 
product description. 

d) Consistency 

The programs and data shall be free from contradic- 
tions within themselves and with the product descrip- 
tion and the user documentation. Each term should 
have the same meaning everywhere. 

The control of the program operation by the user and 
the program behaviour (for example, messages, input 
screen formats and printed reports) should be uni- 
formly structured. 

3.3.2 Reliability 

The system, comprising hardware, required software 
and those programs that belong to the product, shall 
not fall into such a state that the user cannot; control it, 
and shall neither corrupt nor lose data. 

This requirement shall even be met in the case that 

- capacity is exploited up to the specified limits; 

- attempts are made to exploit capacity beyond 
the specified limits; 

- an incorrect input is made by the user or from 
another of the programs listed in the product de- 
scription; 

- explicit instructions in the user documentation 
are violated. 

Only those possibilities for hardware and operating 
system interruption that cannot be caught by any pro- 
gram (for example, the key or combination of keys for 
system operation reset) are excluded. 

The programs shall recognize violations of syntactic 
conditions for input. In the case where a program 
recognizes an input as erroneous or as undefined, it 
shall not process this as permissible input. 

3.3.3 Usability 

With respect to usability, parties to agreements based 
on this International Standard are encouraged to in- 
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vestigate the possibility of applying the most recent 
editions of standards in the ISO 9241 series. 

Note - In particular parts 1 and 13 of the ISO 9241 series should be 
considered. 

a) Uniderstandability 

The questions, messages and results of the programs 
should be understandable, for example, 

- by an adequate selection of terms; 

- by graphical representations; 

- by provision of background information; 

- by the explanations of a help function. 

Error messages shall offer detailed information ex- 
plaining the cause or the correction of the corre- 
sponding usage errors (for example by a reference to 
an Item in the userdocumentatron). 

b) Ease of overview 

Each data medium shall bear the product identification 
and, if there is more than one medium, a distinguish- 
ing number or text. 

For the user, when working with the programs it shall 
always, be possible to find out which function is being 
executed. 

The programs should provide information to the user 
in such a form that it is easily visible and easy to read. 
The user should be guided by appropriate coding and 
grouping of the information. Where necessary, the 
programs should alert the user. 

Messages from the programs should be so designed 
that the user can easily differentiate them by type, for 
example, 

- acknowledgement; 

- queries from programs; 

- warnings; 

- error messages. 

Input screen formats, reports and other inputs and 
outputs shxDuld be designed to be clear and easy to 
overview. Possibilities for this include 

- alphanumeric fields are left-aligned; 



- numeric fields are right-aligned; 

- in tables, decimal points or commas are ar- 
ranged in the same vertical line; 

- field limits are recognizable; 

- fields, the use of which is obligatory, are rec- 
ognizable as such; 

- identified input failures are immediately high- 
lighted in the input screen format; 

- the user's attention is directed to a change in 
screen content by a visual or audible sign. 

c) Operability 

The execution of functions that have serious conse- 
quences shall be reversible, or the programs shall give 
a clear warning of the consequences and request 
confirmation before executing the command. In 
particular, erasure and oven/vriting of data, as well as 
interruptions of a lengthy processing operation, have 
serious consequences. 

if documenting text is offered in the dialogue, the user 
should be able to access subclauses of the text in a di- 
rect manner, for example, by selection from a dis- 
played table of contents and by a search function 
based on keywords. 

3.3.4 Efficiency 

Nothing is required. However, statements on efficien- 
cy in the product description shall be conformed to. 

3.3.5 Maintainability 

Nothing is required. However, statements on main- 
tainability in the product description shall be con- 
formed to. 

3.3.6 Portability 

Nothing is required. However, statements on portabil- 
ity in the product description shall be conformed to. 



4 Instructions for testing 

The instructions in 4.1 to 4.5 specify how a product 
shall be tested against the quality requirements. They 
include both testing for properties required from all 
conforming products and testing for properties 
promised by the product description. Thay include 
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both testing by inspection of documents and black- 
box testing of programs and data. 

These instructions describe functional testing (black- 
box testing). Structural testing is not included be- 
cause it would require the availability of the source 
code. 

Only the product in its required systems is tested. The 
ergonomic evaluation at the computer workplace is 
not considered in this International Standard. 

NOTES 

1 These instructions are primarily aimed at third party testing 
according to some certification scheme (see clause 1, item c). During 
production it may be cheaper and more effective to use structural 
testing. 

2 Clause 4 does not contain requirements on software packages (all 
these are contained in clause 3). A software package may conform 
without having been tested according to clause 4, and such a test may 
fail to discover an existing non-conformity. 

3 As the required system is determined by the product description, 
any nonconformity of the product on the required system Is treated as a 
nonconformity of the product. 

4 A certification scheme may make testing against recommen- 
dations optional. 

5 Guidance on ergonomic evaluation is contained In ISO 9241-1 1 . 



4.1 Test pre-requisites 

4.1.1 Presence of product items 

For testinn a software rsackaoe all items to be deliv- 
ered (see 3.1.2 h) as well as the requirements docu- 
ments identified in the product description (see 3.1.2 
e) shall be present. 

4.1.2 Presence of system constituents 

For testing a software package it is required that the 
constituent parts of all the computer systems as 
named in the product description are available. 

4.1.3 Training 

If training is mentioned in the product description, the 
tester shall have access to the training material and the 
training program. 



4.2 Testinrg activities 

The product description, user documentation, pro- 
grams and any data to be delivered as parts of the soft- 
ware package 

- shall be tested for compliance with the require- 
ments in clause 3, and 

- should be tested for compliance with the re- 
commendations in clause 3. 

The test objectives shall be derived from, and include 
all, the requirements in clause 3 (completeness, con- 
sistency etc.). 

If other products are mentioned in the product de- 
scription, these other products need only be tested 
for the claims made in the product description of the 
product under test. 

Details in the product description, in the user docu- 
mentation, in functions or in data of the product do not 
need to be tested if according to the judgement of the 
tester 

- they have negligible influence on the syit- 
ability for the named work task, and 

- they can be tested in principle but not with jus- 
tifiable expenditure. 

Such details which are not tested shall be mentioned 
in the test records and in the test report. The reasons 
for not testing them shall be documented in the test 
records. 

4.2.1 Product description 

The fulfilment of the requirements in clause 3 shall be 
tested, and the fulfilment of the recommendations in 
clause 3 should be tested. 

4.2.2 User documentation 

The fulfilment of the requirements in clause 3 shall be 
tested, and the fulfilment of the recommendations in 
clause 3 should be tested. 

4.2.3 Programs and data 

The fulfilment of the requirements in clause 3 shall be 
tested, and the fulfilment of the recommendations in 
clause 3 should be tested. 
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The programs shall be tested in all the computer 
systems that are named in the product description. 

If there are several program variants, each shall be 
tested. Functions that, according to the product 
description and user documentation, are identical in a 
number of variants may be tested each in one variant. 

The programs and the data supplied with them shall 
be tested using test cases constructed on the basis of 
the product description and the user documentation. 
Further material (for example, source programs) need 
not t>e considered unless the testing of statements in 
the product description or user documentation makes 
this necessary. 

The test cases shall be constructed methodically and 
systematically. 

NOTE - Methodical random testing is permitted. 

If examples are given in the user documentation they 
shall be used as test cases, but testing shall not be re- 
stricted to these examples. 

Test cases provided by the supplier of the software 
package may be used but testing shall not be restric- 
ted to these test cases. 

a) Installation 

If, according to the product description, installation 
can be carried out by the user, it shall be tested that 
the programs can be installed and tested for 
successful installation as described in the installation 
manual. 

Otherwise it shall be ensured that the hardware and 
software environment of the installed programs corre- 
sponds to the statements in the product description 
for the computer system under consideration. 

b) Program execution 

The test cases shall cover all functions described in 
the product description and user documentation and 
they shall take into account combinations of functions 
that are representative for the work task. 

The programs shall be tested for all boundary values 
(according Ao product description and user documen- 
tation) in the required systems for which these values 
apply. 

Input or command sequences that are explicitly depre- 
cated or declared as forbidden in the user documenta- 
tion (see 3.3.2) shall be Used in the testing. 



4.3 Test records 

The records for each test shall contain sufficient infor- 
mation to permit the repetition of the test [ISO/IEC 
Guide 25]. They shall include 

- a test plan or test specification containing test 
cases (each test case stating its objectives, see 
2.6); 

- all the results associated with the test cases, 
including all the failures occurring during the test; 

- the identity of personnel involved in testing. 



4.4 Test report 

The objects and the results of the test (as recorded in 
the test records) shall be summarized in a test report. 
The test report shall have the following structure: 

1 Product identification 

2 Computer systems used for testing (hardware, 
software and their configuration) 

3 Documents used (with their identifications) 

4 Results of the teat of the product description, 
user dpcu mentation, programs and data 

5 List of the non-conformities to requirements 

6 Either a list of the non-conformities to recom- 
mendations, or a list of the recommendations that 
were not followed, or a statement that the product 
was not tested for conformity to recommendations 

7 Date of the termination of the test 

Chapter 4 of the test report (Results of the test) shall 
contain a statement corresponding to each headline 
of 3.1 to 4.2. 

In addition to the statement that the product was not 
tested for conformity with recommendations, chapter 
6 of the test report may provide a list of observed non- 
conformities to recommendations. 

The identification of the test report (testing laboratory, 
product identification, date of the test report) and the 
total number of its pages shall appear on each page of 
the test report. 
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The test report shall include 

- a statement to the effect that the test results 
relate only to the items tested; 

- a statement that the test report shall not be re- 
produced except in full without the written approv- 
al of the testing laboratory [ISO/IEC Guide 25]. 

The test report should comply with the provisions for 
test reports in ISO/IEC Guide 25. 



4.5 FoHowuptest 

When a product, which has already been tested, is 
tested again (taking into consideration the previous 
test), then 

- all changed parts in the documents, functions 
and data shall be tested as if it were a new product; 

- all unchanged parts that are expected to be in- 
fluenced by the changed parts or by changes in a 
required system (according to the specialized 
knowledge of the tester) shall be tested as if it 
were a new product; 

- all other parts shall at least be tested by 
samples. 
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Annex A 

(informative) 

Definitions from otiier standards 



For easy reference, definitions are quoted here for 
some terms used in this International Standard but de- 
fined in other standards. At the time of publication, the 
editions indicated were valid, and the definitions quo- 
ted from them were used or considered. 



A.1 General terms 

A.1.1 software: All or part of the programs, proce- 
dures, rules', and any associated documentation of an 
information processing system. [ISO/IEC 2382- 
1:1993, without the Note] 

A.I. 2 software package: A complete and docu- 
mented set of programs supplied to several users for a 
generic application or function. [ISO/IEC 2382- 
20:1990, without the Note] 

A.I. 3 system software: Application-independent 
software that supports the running of application soft- 
ware. [ISO/IEC 2382-20:1990] 

A.I. 4 utility routine, utility program: A routine [A 
computer program] that provides general, frequently 
needed services for computer users and service per- 
sonnel. [ISO 2382-7:1989, without the examples] 

A.I .5 functional unit: An entity of hardware or soft- 
ware, or both, capable of accomplishing a specified 
purpose. [ISO/IEC 2382-1:1993] 

A.I .6 (computer) program: A syntactic unit that con- 
forms to the rules of a particular programming lan- 
guage and that is composed of declarations and state- 
ments or instructions needed to solve a certain func- 
tion, task, or problem. [ISO/IEC 2382-1:1993] 

A. 1.7 interface: A shared boundary between two 
functional units, defined by functional characteristics, 
common physical interconnection characteristics, sig- 
nal characteristics, and other characteristics, as ap- 
propriate. [ISO 2382-9:1984, without the Note] 

A.I .8 user interface: An interface that enables infor- 
mation to be passed between a human user antf hard- 
ware or software components of a computer system. 
[ANSI/IEEE Std 610.12-1990] 



A.I .9 configuration: The manner in which the hard- 
ware and software of an information processing 
system are organized and interconnected. [ISO/IEC 
2382-1:1993] 



A.2 Characteristics of a product 

A.2.1 functionality: A set of attributes that bear on 
the existence of a set of functions and their specified 
properties. The functions are those that satisfy stated 
or implied needs. [ISO/IEC 9126:1991, without the 
Notes] 

A.2.2 reliability: A set of attributes that bear on the 
capability of software to maintain its lev^l of perfor- 
mance under stated conditions for a stated period of 
time. [ISO/IEC 9126:1991, without the Notes] 

A.2.3 usat^ility: A set of attributes that bear on the 
effort needed for use, and on the individual assess- 
ment of such use, by a stated or implied set of users. 
[ISO/lEC 9126:1991, without the Notes] 

A.2.4 efficiency: A set of attributes that bear on the 
relationship between the level of performance of the 
software and the amount of resources used, under 
stated conditions. [ISO/IEC 9126:1991, without the 
Note] 

A.2.5 maintainability: A set of attributes that bear 
on the effort needed to make specified modifications. 
[ISO/IEC 9126:1991, without the Note] 

A.2.6 portability: A set of attributes that bear on the 
ability of software to be transferred from one environ- 
ment to another. [ISO/IEC 9126:1991, without the 
Note] 



A.3 Data 

A.3.1 data: A reinterpretable representation of infor- 
mation in a formalized manner suitable for communica- 
tion, interpretation, or processing. [ISO/IEC 2382- 
1:1993] 

A.3.2 data medium: A material in or on which data 
can be recorded and from which data can be retrieved. 
[ISO/IEC 2382-1:1993] 
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A.4 Testing 

A.4.1 test: Technical operation tiiat consists of tlie 
determination of one or more characteristics of a given 
product, process or service according to a specified 
procedure. [ISO/IEC Guide 2:1991] 

A.4.2 test data: The data used for a check problem. 
tISO 2382-8:1986] 

A.4.3 check problem: A problem with a known solu- 
tion used to determine whether a functional unit is op- 
erating correctly. [ISO 2382-8:1986] 

A.4.4 test method: Specified technical procedure 
for performing a test. [ISO/IEC Guide 2:1991] 

A.4.5 test plan, system test and evaluation plan: 

A plan that establishes detailed requirements, criteria, 
general methodology, responsibilities, and general 
planning for test and evaluation of a system. [ISO/IEC 
2382-20:1990] 

A.4.6 test report: Document that presents test- re- 
sults and other information relevant to a test. [ISO/IEC 
Guide 2:1991] 



A.5 Other terms 

A.5.1 program maintenance manual: A document 
that provides all the information necessary to maintain 
a program. [ISO/IEC 2382-20:1990] 

A.5.2 system maintenance: The modification of a 
system to correct faults, to improve performance, or 
to adapt the system to a changed environment or 
changed requirements. [ISO/IEC 2382-20:1990] 

A.5.3 work task: An intended outcome of the work 
system. [ISO 6385:1981] 

A.5.4 work system: The work system comprises a 
combination of people and work equipment, acting to- 
gether in the work process, to perform the work task, 
at the work space, in the viork environment, under the 
conditions imposed by the work task. [ISO 
6385:1381] 
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Annex B 

(informative) 

Example of a product description 



The following example describes, in accordance with 
this International Standard, a simple imaginary soft- 
ware package, in order to show information that shall 
be present in every product description. 

Product description sheet 

F^REatWORK Version 2.6 

FIREatWORK - screen saver and password protection 

The program FIREatWORK will save your screen by 
displaying a stunning - and on Red-Green-Blue 
screens colourful - firework while you are not working 
with your computer. If you enter a password you will be 
warned if anyone has tampered with your computer in 
your absence. 

FIREatWORK is installed in the memory. It will activate 
itself when you don't press any key and don't move 
the mouse (if any) for some (adjustable) time. It will 
stop as soon as you press any key or move the 
mouse. However, if you define a password, 
FIREatWORK will wait for that password to be typed. 

You can define your favourite settings for 

- the time FIREatWORK will wait before activat- 
ing itself (1 to 999 minutes, or never); 

- the number of fireworks that will explode to- 
gether (1 to 19). 

For this, FIREatWORK will use line dialog or a window 
(as your operating system does for changing system 
date and time). 

In the same way you may define a password (6 to 45 
characters). Then, if FIREatWORK stops upon typing 
an arbitrary character or does not stop upon typing 
your password, someone has Interrupted FIRE- 
atWORK (e.g. by switching off the power) and re- 
started it without a password or with a different one. 

You may produce backup copies of the program and 
the setting by means of your operating system. The 
password is not saved. 

Some technical details: 



- FIREatWORK runs on a Quince Hardcore 
119xi personal computer (and on compatible 
computers) with at least 1MB main memory and a 
90 mm (3,5 in) or 130 mm (5,25 in) diskette drive 
for at least 720 KB. It does not need a hard disk. It 
supports a serial or parallel Mini-RAT mouse (or 
any other mouse, if compatible), but no mouse is 
required; 

- FIREatWORK needs a graphics card: Hercules 
DeLuxe or PowerEGA 16+ (or any other card, 
if compatible); 

- FIREatWORK runs under B.I.T.S 1.01 or 
Gnome 3,0 (or any operating system compatible 
with one of these two). 

When ordering FIREatWORK please tell us 

- whether you want the variant for B.I.T.S or the 
one for Gnome; 

- whether you want FIREatWORK on a 90 mm 
(3,5 in) diskette or on a 130 mm (5,25 in) diskette. 

The packet consists of the program (load module) on 
one diskette and a documentation booklet which in- 
cludes the installation guide. 

Important for you: 

- You don't need any special knowledge to 
install or use FIREatWORK. 

- Program messages and documentation are 
written in English. 

- FIREatWORK fully complies with ISO/IEC 
12119:1994 Information technology - Software 
packages - Quality requirements and testing. 

Neither support for operating the product nor mainte- 
nance is offered. 

Where do you get FIREatWORK: 

PyroManiac Klaus P Schmidt Ltd 
33 Bell Street 
Bergheim, SU 53844 
Telephone (022) 845 3902 
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